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(54) Electro-optic reconnaissance 



(57) To perform long range, sector scan panoramic electro-optical reconnaissance of a scene, e.g. from an 
aircraft (102), at increased aircraft velocities, the scene is scanned at a non-linear scan velocity. The effects of 
the non-linear scan velocity are electronically removed to correct the aspect ratio of the final image. As 
described, a focal plane array (412) detects an image of the scene, and the image is converted into an 
electronic signal which is a digital representation of the image. A lens arrangement is used to focus a narrow 
slit of the scene onto the focal plane array, and a rotating prism scans the slit across the scene at a non-linear 
scanning velocity. The electronic signal is transmitted to a ground station (402) where it is processed to 
provide visual image data that represents the scene. Distortion induced into this data as a result of the 
non-linear scan velocity is removed (434) to provide a distortion free final image. 
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ELECTRO-OPTIC RECONNAISSANCE SYSTEM AND METHOD 



A portion of the disclosure of this patent document contains material 
which is subject to copyright protection. The copyright owner has no objec- 
tion to the facsimile reproduction by anyone of the patent disclosure, as it 
appears in the Patent and Trademark Office patent files or records, but other- 
10 wise reserves all copyright rights whatsoever. 



The present invention relates generally to electro-optical reconnaissance 
systems and methods. 

15 

Electro-optical systems enjoy widespread use in contemporary 
reconnaissance systems. There are three primary reasons for this popularity. 
The first reason is that these systems are able to operate in real time. In other 
20 words, these systems are able to process and interpret data as it is collected. 
These systems collect data using an airborne camera system, transmit the data 
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to a ground station via an air-to-ground data link, and process the data at the 
ground station as it is received. This allows data to be interpreted much more 
quickly than similar data recorded on photographic film, flown back to a home 
base, and processed subsequent to the flight operation. 

5 A second advantage is the ability of the electro-optical system to 

penetrate haze. This ability is made possible by signal processing techniques 
which are able to separate and enhance the data information from background 
noise (haze). This ability does not exist with conventional photographic 
reconnaissance techniques since it is not possible to remove the effects of 

10 background noise. 

Additionally, because of the high sensitivity of electro-optical detectors, 
electro-optical systems can operate with less ambient light than photographic 
systems. This has the effect of extending the amount of time per day during 
which a reconnaissance mission can be flown. 

15 There are typically two general forms of electro-optical reconnaissance 

systems. FIG. 1 illustrates the two general forms of electro-optical 
reconnaissance systems. In a first mode, called a strip mode system, the area 
detected by the electro-optical system is a long, narrow slit which can be 
described as a projection of a slit 104. The projection of slit 104 is the area 

20 detected (projected) by a focal plane array (FPA) of the system. The FPA is 
mounted in an aircraft 102. A lens arrangement is used to focus slit 104 onto 
the FPA. Typically, the FPA is a line of optical sensor devices such as 
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CCDs. In a strip mode system, the projection of the slit 104 extends at right 
angles to the direction of flight and constitutes one dimension of the image. 
The direction of flight is shown by a flight path 122. The second dimension 
of the image is generated by the forward motion of aircraft 102 as it flies 

5 along flight path 122 at a velocity V. 

In this specification, the direction of forward motion of the aircraft will 
be referenced as the in-track direction. The direction at a right angle to the 
flight path is referred to as the cross-track direction. 

The second mode is a sector scan panoramic mode (sector scan mode). 

10 In the sector scan mode, the line of detectors in the FPA is aligned in the in- 
track direction. Hence, a projection 106 of the FPA is in the in-track 
direction. Projection 106 is scanned at a right angle to the flight path (the 
cross-track direction) across the scene to be imaged. Scanning in the cross- 
track direction provides the second dimension of the image. 

15 For long-range aerial surveillance applications, long-range oblique 

photography (LOROP) systems are utilized. A typical LOROP system uses 
an aircraft-mounted electro-optical camera configured to scan a scene at or 
near the horizon in the sector-scan mode. The scanned objects are focused by 
a lens or other optics onto the FPA. A rotating prism may be used to scan 

20 projection 106 in the cross-track direction across the scene to be sampled. 
The FPA is often one picture element (pixel) high and several thousand pixels 
wide. 
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The electro-optical camera generates an electronic signal that represents 
an image of the scene scanned. This signal is downlinked to a ground station 
where it is converted into visual information. 

Referring again to FIG. 1, a LOROP system will be described in more 

5 detail. An airplane 102 flies at an altitude A above the ground and at a ground 
distance D from the scene to be photographed. Airplane 102 travels at 
velocity V in the in-track direction parallel to the scene. The line-of-sight 
distance between airplane 102 and the scene is defined as a slant range p slant . 
For long-range (LOROP) applications, slant range p slant is large. (For 

10 example, in a typical application, p slant can be on the order of 40 nautical 
miles). 

The FPA and associated optics are mounted in airplane 102. A 
depression angle fi d is defined as the angle of the camera's line-of-sight with 
respect to a horizontal plane. A rotating camera barrel causes projection 106 
15 to be scanned across the scene at or near the horizon. Velocity V of airplane 
102 in the in-track direction and parallel to the scene causes the camera to 
photograph adjacent slices of the scene. Each adjacent slice forms a complete 
picture. 

FIG. 2 illustrates these scanned slices in more detail. Referring to 
20 FIGs. 1 and 2, the length of each slice is determined by the distance covered 
by the scanning motion of the camera in the cross-track direction. This length 
is referred to as a cross-track field-of-coverage 202. The width of each slice 
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in the in-track direction is defined by the focal plane array width, the focal 
length of the optics, and the distance between the camera and the scene. This 
width is known as the in-track field-of-coverage 204. The slices overlap each 
other in the in-track direction by an amount known as a forward overlap 206. 

5 Forward overlap 206 ensures that no part of the scene is left unscanned. 

In-track field-of-coverage 204 is a function of the p $bnt . According to 
lens arrangements typically employed, in-track field-of-coverage 204 is larger 
(larger on the ground, but the same angular coverage) at the far end of the 
scan (far-field point of scan) than it is at the point of scan closest to the 

10 aircraft (near-field point of scan). This phenomenon is not illustrated in FIG. 
2 for simplicity. Instead, FIG. 2 illustrates an in-track field-of-coverage 204 
as the same for both the near-field and the far-field point of scan. 

A vertical scan velocity (in the cross-track direction) is selected so that 
for a given airplane 102 velocity V, a specified amount of forward overlap 206 

15 is obtained. The amount of forward overlap 206 specified is chosen so that 
no image information is missed between scans. As the velocity Vof airplane 
102 increases, the vertical scan velocity must also increase to maintain the 
specified amount of forward overlap 206. 

To obtain optimum system resolution, the information in the FPA must 

20 be read each time the vertical scan causes the FPA to traverse the area 
projected by each pixel. As more area is detected (projected) by a pixel 
between FPA reads (i.e., as scanning velocity increases), system resolution 
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diminishes. Thus, to maintain system resolution, as the vertical scan velocity 
increases, the rate at which the information in the FPA is read must increase 
as well. Because the rate a which the information in the FPA may be read is 
limited by detector technologies, the vertical scan velocity is limited to a 
5 practical maximum rate. Since vertical scan velocity is bounded to a practical 
maximum by the maximum FPA read rate, and since an increase in vertical 
scan velocity is required to maintain a specified forward overlap 206 as 
airplane 102 velocity V increases, velocity Vis limited to a maximum. Thus, 
for a given resolution and forward overlap, the operational velocity Vat which 

10 sector-scan panoramic reconnaissance can be performed is limited to a 
maximum. Conventional LOROP systems have not been able to overcome this 
limitation without sacrificing resolution. 

Note that in this document, system resolution is defined in terms of 
some constant number of line pairs per unit length on the ground. 

15 What is needed is a system and method for increasing the operational 

velocity Vat which an aircraft performing electro-optical reconnaissance can 
fly while maintaining a specified forward overlap 206, and maintaining system 
resolution to a given level. 



In accordance with one aspect of the present invention, a 
system for performing long range, sector scan panoramic 
electro-optical reconnaissance of a scene, the 
reconnaissance performed at increased scan velocities for 
a given ground resolved distance and forward overlap 
comprises : 

(1) a focal plane array configured to detect an image 
of the scene and to convert said image into an 
electronic charge representation of said image; 

(2) a main electronics unit, coupled to said focal 
plane array, configured to convert said 
electronic charge representation into a digital 
image data signal, wherein said digital image 
data signal is a digital representation of said 
image; 

(3) first means, coupled to said focal plane array, 
for focusing a portion of the scene onto said 
focal plane array, wherein said portion of the 
scene is defined by a projection of said focal 
plane array; 

(4) second means, coupled to said first means, for 
scanning said projection of said focal plane 
array across the scene at a non-linear scan 
velocity; 

(5) third means, coupled to said second means, for 
determining said non-linear scan velocity; and 

(6) fourth means, coupled to said main electronics 
unit, for processing said digital image data 
signal to provide a visual image signal 
representing a corrected image, said corrected 
image having a pixel aspect ratio corrected to 
remove the effects of said non-linear scan 
velocity. 

In accordance with a second aspect of the present 
invention, a system for performing long range, sector scan 
panoramic electro-optical reconnaissance of a scene from an 
airborne craft, the reconnaissance performed at increased 
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scan velocities for a given ground resolved distance, and 
forward overlap comprises: 

first means for detecting an image of a portion of the 
scene and converting said image into a digital image data 
signal, wherein said image data signal is a digital 
representation of said image; 

second means, coupled to said first means, for 
focusing said portion of the scene onto said first means, 
and for scanning said portion of the scene across the 
entire scene at a non-linear scan velocity; and 

third means, coupled to said first means, for 
processing said digital image data signal to provide a 
visual image data signal representing a corrected image, 
said corrected image having a pixel aspect ratio corrected 
to remove the effects of said non-linear scan velocity. 

In accordance with a third aspect of the present 
invention, a method for optimizing long range, sector scan 
panoramic electro-optical reconnaissance of a scene from an 
airborne craft comprises the steps of: 

(1) determining a camera scan rate and a focal plane 
array read rate for a desired level of 
operational performance; 

(2) determining a non-linear scan velocity if said 
scan rate is above a threshold level; 

(3) scanning the camera across the scene at said non- 
linear scan velocity determined in step (2) ; 

(4) focusing an image of the scene scanned in step 
(3) onto a focal plane array and converting the 
image into an electronic signal that represents 
the scene; 

(5) processing said electronic signal to generate a 
digital image of the scene, wherein said 
processing comprises a step of correcting a pixel 
aspect ratio of said digital image to remove 
effects of said non-linear scan velocity. 



9 



The present invention is an apparatus and method for extending the 
operational velocity of sector-scanning panoramic LOROP without sacrificing 
system resolution. 

5 The present invention is directed toward a system and method for 

increasing the scan velocity of a sector scan electro-optic reconnaissance 
system while maintaining a specified level of forward overlap and system 
resolution as defined above. The present invention takes advantage of 
increased resolution inherent in the near field portions of the scan. As the 

10 camera scans from the far field to the near field, the scan velocity is 
increased. This increase in scan velocity allows the system to trade increased 
near-field resolution in exchange for an increased scan velocity. Increasing 
the scan velocity in the near field results in an overall increase in the scan 
rate, while maintaining a specified level of system resolution. This increase 

15 in the scan rate allows the aircraft to operate at a greater velocity without 
sacrificing forward overlap. 

In one example, calculations are performed to 

determine the desired camera scan rate given the operational parameters of the 
mission. An associated FPA read rate, required to meet performance 

20 specifications (particularly resolution) at the determined scan rate, is 
calculated. For faster camera scan rates, the FPA read rate must also be 
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faster. At camera scan rates above a threshold value, the FPA must be read 
at a rate faster than the detector technology permits. If the calculated FPA 
read rate exceeds the system maximum, then the scan velocity will have to 
vary as a function of time. 

In the case of a varying scan rate, the scan velocity is at threshold at 
the far field point of the scan and increases as the scan progresses through the 
near field. This increase results in an increase in the overall scan velocity of 
a camera for the scan cycle. The non-linear scan velocity used throughout 
each scan is determined using either an exact solution or a polynomial 
approximation. 

The camera is scanned across the scene at the non-linear velocity 
determined above. Thus, an image of the scanned scene is focused onto the 
focal plane array. Data are read out of the focal plane array at periodic 
intervals thus causing strips of the scanned scene to be electronically 
photographed. This data is an electronic signal comprising digital image data. 
The signal is sent to a ground station for processing to ultimately obtain a 
visual image. 

However, scanning the scene at a non-linear velocity causes the pixel 
aspect ratio of the resultant image to be distorted, and to vary throughout the 
image. To compensate for this, the digital image data are corrected to remove 
the effects of the non-linear scanning velocity. Once corrected, a distortion 
free visual image can be produced having a pixel aspect ratio of 1:1 
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throughout the entire image. Thus, the present invention provides improved 
electro-optical reconnaissance system performance by using a non-linear 
scanning velocity that takes advantage of increased near field resolution, and 
by then electronically correcting the resultant image data to remove the effects 

5 of the non-linear scanning velocity. 

Further features and advantages of the present invention, as well as the 
structure and operation of other embodiments of the present invention, are 
described in detail below with reference to the accompanying drawings. In the 
drawings, like reference numbers indicate identical or functionally similar 

10 elements. Additionally, the left-most digit of a reference number identifies the 
drawing in which the reference number first appears. 



The present invention will be described with reference to the 
accompanying drawings, wherein: 
15 FIG. 1 is a diagram illustrating sector-scan electro-optical 

reconnaissance system. 

FIG. 2 illustrates photographic slices 106 from a sector-scan electro- 
optical reconnaissance system. 
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FIG. 3 is a flow chart illustrating a method according to the present 
invention. 

FIG. 4 is a high level block diagram illustrating key elements of the 
present invention and its environment. 

FIG. 5 is a flow chart illustrating the steps involved with determining 
the desired scan velocity. 

FIG. 6 is a flow chart illustrating the steps required to determine a scan 
time used. 

FIG. 7 is a block diagram illustrating elements of the present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 
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1. Overview of the Invention 



The present invention is a system and method for non-linear scanning 
in electro-optical reconnaissance systems to allow an increased maximum 
operational aircraft velocity for a specified level of system resolution and 
forward overlap. The present invention takes advantage of increases in near- 
field resolution by increasing the scan velocity in the near field while 
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maintaining a given FPA read rate (typically the maximum rate). Thus the 
overall scan velocity is increased without sacrificing system resolution. Since 
the overall scan velocity is increased, aircraft velocity V can be increased 
while maintaining a specified forward overlap. 

With the FPA read rate held constant, as the scan velocity increases 
more area of the scene in the cross-track direction is imaged per FPA read. 
Because the scan velocity increases as the system scans the near field, a 
greater area of the scene is imaged between FPA reads at the near field, than 
at the far field. As a result, the angular aspect ratio of the pixels within a 
single image is not a constant 1:1. Thus, the resultant image is distorted. The 
image is elongated in the cross-track direction (with respect to the in-track 
direction). The amount of elongation increases (in the near field) as the scan 
velocity increases. The pixels are actually elongated in the cross-track 
direction in that their angular dimension is larger. However, when the image 
is viewed it appears "squashed." The squashed appearance is more 
pronounced in the near field. 

To provide a final product image having pixels with an angular aspect 
ratio of 1:1 (i.e., distortion free) everywhere within the image, the image is 
corrected during image processing. 

Thus the present invention provides improved electro-optical-LOROP 
system performance by performing a non-linear scan to take advantage of 
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increased near-field resolution, and by electronically correcting the image data 
to correct for the non-linear scan rate. 

2. Terminology 

Table 1, below, outlines the definitions of terminology and symbols 
used in this application. The terms in Table 1 are ordered so that each is 
defined only in terms of those terms that have already been defined in the 
table. 

Some of the terms listed in Table 1 have been previously defined in 
this application. The definition of these terms is repeated in Table 1. 

In this application, the terms 'scan rate' and 'scan velocity' are used 
interchangeably. These terms refer to the angular velocity at which the 
camera is scanned. 



Table 1 



Term 


Symbol 


Definition 


velocity 


V 


Velocity of the aircraft relative to 
the ground (nmile/hour). 


height 


H 


Altitude of the aircraft above the 
ground (feet). 


in track 


N/A 


Parallel to the direction of the 
flight of the aircraft. 



16 



Term 


Symbol 


Definition 


cross track 


N/A 


Perpendicular to the direction of 
the flight of the aircraft. 


depression angle 


Ad 


Ine angle measured from the 
horizontal at which the camera is 
pointed in the cross-track 
direction. 


minimum 
depression angle 


1 ^dmm 


The minimum depression angle 
that occurs during a given scan of 
an image. 


maximum 
depression angle 




The maximum depression angle 
that occurs during a given scan of 
an image. 


slant range 




The distance from the aircraft to 
the point on the ground at which 
the camera is currently pointing. 


ground range 


Pground 


The distance from the point 
directly below the aircraft on the 
ground to the point on the ground 
at which the camera is currently 
pointing. 


scan velocity 


O) { 


Angular velocity (radians/sec) at 
which the camera scans the ground 
in the cross-track direction 

ddd 
dt ' 


in-track gsd 


• 

Igsd 


The distance (feet) at the ground 

nernendiculpr fn ihp !inp n€ ciaht in 

the in-track direction represented 
by a scanned pixel. 


cross-track gsd 


X GSD 


The distance (feet) at the ground 
perpendicular to the line of sight in 
the cross-track direction 
represented by a scanned pixel. 
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Term 


Symbol 


Definition 


ground sample 
distance 


GSD 


GSD 2 = IgsD * ^GSD | 

Where i GSD and Xq SD differ by 
more than a factor of 2, GSD is 
limited to the greater of the two 
divided bv 2*. 


cycle rime 




The time (seconds) between the 
start of one scan and the start of 

uiC/ iiCAi acaii. 


minimum cycle 
time 


tcmin 


The minimum time (seconds) 
allowed between the start of one 

cr»o n cnH tYif* ct?*rf nf th^ n^vf ^C^lTl 
built allU UIC oUxll Ul UIC HwAL await. 


scan time 


t, 


the time (seconds) actually 
available for scanning an image 
during the cycle time. 


dead time 


td 


The time (seconds) not available 
for scanning an image during the 
cycle time. 


scan time used 


^sused 


The scan time (seconds) actually 
used for scanning an image; an 
integral number of 1 millisecond 

interval c 


line rate 


lr 


The (variable) rate at which lines 
of pixels are generated by the 


maximum line 
rate 




The maximum rate at which lines 
of pixels can be generated by the 
camera system. 


minimum line 
rate 


l^inia 


The minimum rate at which lines 
of pixels can be generated by the 
camera system. 


low-light setting 


7i 


If "off, normal camera operation. 
If "on\ low-light conditions exist; 
calculated line rate lr and 
maximum line rate lr^ are 
affected. 
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Term 


Symbol 


Definition 


exposure control 


E c 


A power of two (may be < 1) by 
which the calculated line rate 
snouiG De multiplied to compensate 
for too much or not enough light. 


forward scan 


^dmax * 
^dmin 


A scan which begins at 0 dmax and 
endsat0 dmin . 


reverse scan 


^dmin " 
^dmax 


A scan which begins at 0 dmin and 
ends at 0 dmax . 


overlap 


ov 


The amount (fraction of LO) by 
which consecutive scanned images 
must overlap on the ground at 
0 dmax ; a function of camera mode. 


in-track 
coverage 


^COV 


The total angle scanned in the in- 
track direction; same for all 
camera modes. 


cross-track 
coverage 


x cov 


The total angle scanned in the 
cross-track direction; fixed for 
camera modes 1, 4, and 5; 
variable for modes 2 and 3. 


focal length 


f 


Focal length (inches) of the camera 
system. 


number of 
detector 
elements 


n det 


the number of detector elements in 
the linear detector array, including 
a "space" for two non-existent 
detectors in the middle of the 
array. 


detector width 


w det 


The width (mm) of a single 
detector element in the pixel (in- 
track) direction. 


detector angle 


^det 


The angle in the pixel (in-track) 
direction "seen" by a detector 
element at the middle of the 
detector array. 



19 



Term 


Symbol 


Definition 


desired linear 
line angle 




The desired angle in the line 
(cross-track) direction "seen" by a 
detector element for a linear scan; 
fl JH s= # . for modes 12 3 and 5 
and 0 dU = 0 d J2 for mode 4. 


number of lines 


n lines 


The number of image lines 
penerated during a scan. 


full resolution 
image line 
number 




The line number (zero relative) of 
a line in the full resolution image 
collected during a scan. 


full resolution 
line group 


nig 


The number of full resolution lines 
corresponding to a given line in 
the minified view image. 


cutoff 
depression angle 


^dcutoff 


The depression angle at which we 
switch from constant x^ to 
variable x GSD during a mode 2 or 
mode 3 scan. 


cuttoff time 


^cutoff 


The time corresponding to 0 dcuto(r at 
which we switch from constant 
Xgsd to variable Xq SD during a 

mnHp 9 or mrvte ^ cr*pn 


end-of-frame 
time 


tend 


The time elapsed after the last 
complete image line has been 
generated during the scan. 


minified view 
size 


n minif 


The dimension in pixels of the 
iW x n^ pixel buffer used to 
store minified view images. 


minified view 
! image line 
number 


ImmSf 


The line number (zero relative) of 
a line in the minified view image. 
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Term 

X vl MIA 


Svmhnl 


T^ofin it inn 


minified view 
averaging factor 




Each n avg by n^ block of pixels in 
a full resolution image collected 
with a constant angular scan 
velocity will tie averaged to form 
one pixel in the minified view 
image. 


line group angle 




The angle "seen" by a pixel in the 
minified view image. 



3. The Invention and its Environment 

The present invention was developed for use with the F-979H long- 
range tactical electro-optical sensor system, developed by Loral Fairchild 
Systems, Syosset, New York. This sensor system can be mounted in a variety 

5 of aircraft or in a reconnaissance pod or other such airborne craft. The core 
of this system is a Systems Imaging Sensor, comprising an imaging LRU (line 
replaceable unit) and three electronics LRUs. Additional equipment may 
include a reconnaissance management unit interfacing with the aircraft, a 
control panel, optical sights, and an in-flight data recorder. A ground data 

10 system, referred to as an EO-LOROPS ground station, is used to process the 
image data in real time, provide visual displays of the image data, record 
digital data on recorders, and record visual images on film. 



21 



FIG. 4 is a high-level block diagram illustrating a representative 
environment of the present invention. Typical airborne components according 
to the present invention can be mounted within aircraft 102, or within a 
reconnaissance pod. The airborne segment comprises electro-optics 
components 412, in-flight recorders 414, and an air-to-ground transmitter 416. 
In one embodiment, air-to-ground transmitter 416 is not used. Data are stored 
on board the aircraft and delivered to the ground station. 

Electro-optics components 412 comprise an FPA, optics, focus and 
exposure control, optional data compression, computer hardware and software, 
and processing electronics. Electro-optics components receive optical 
information of the scene scanned and provide an electronic signal 424 
representing the optical scene. Electronic signal 424 is recorded using an 
optional in-flight recorder 414. 

Air-to-ground transmitter 416 is used for transmitting scanned image 
information from aircraft 102 to a ground station 402. Such transmission is 
accomplished by a air-to-ground. data link 422. Details of blocks 412-416 will 
be described below in conjunction with FIG. 7. 

Ground station 402 comprises a data link receiver 432, digital image 
data processing 434, and a display 436. Data link receiver 432 receives image 
data on a carrier via air-to-ground data link 422, removes the carrier, and 
forwards the remaining digital image data for processing via an electronic 
signal 426. Optionally, the digital image data are recorded at the output of 
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receiver 432. Such recording is performed for archival purposes or for post 
real-time (off-line) processing. 

Digital image data processing 434 can be configured to provide a 
plurality of data processing functions. The overall goal of digital image data 
processing is to convert digital image data into a useable visual image. 

An alternative environment can be considered wherein digital image 
processing is performed in the aircraft/pod as opposed to in ground station 
402. An additional alternative environment can be considered wherein 
electronic signal is not transmitted via air-to-ground data link 422, but is 
instead recorded on transportable media. In the environment, electronic signal 
424 is retrieved from the media and processed subsequent to the flight 
operation. These alternatives obviate the need for an air-to-ground data link 
and its associated hardware. 

4. Non-Linear Scanning 

LOROP system performance is often specified in terms of an Image 
Inteipretability Rating Scale (IIRS) value. The IIRS value is a quantitative, 
though partly subjective, measure of image quality. It is a function of slant 
range p $!am , altitude A y system resolution, atmospheric visibility , and solar 
illumination. A particular IIRS value is typically defined as a range of ground 
resolved distances (GRD) at a given slant range R anywhere within a given 
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frame. Typical IIRS rating values and their associated GRD are listed at pages 
1-7 of the attached Appendix. 

GRD is defined as the minimum test target element resolved on the 
ground. Generally the system has to sample the ground at two times the 
GRD. Thus, the ground sample distance is 

GRD = 2*GSD 

In other words, the ground must be sampled every .5 feet to resolve a target 
the size of 1 foot under worst-case conditions. 

To meet a given IIRS value, it may be assumed that the scan velocity 
of the camera is limited to a rate at which the camera traverses the distance 
of only one IIRS-specified GSD between each FPA read cycle. In other 
words, for vertical resolution to meet an IIRS requirement, the information in 
the FPA must be read each time the vertical scan causes the FPA to traverse 
a vertical distance of one-half the IIRS-specified ground resolved distance. As 
the vertical scan velocity increases, the rate at which the information in the 
FPA is read must increase as well. Because the rate a which the information 
in the FPA may be read is limited by detector technologies, the vertical scan 
velocity is limited to a practical maximum rate. 
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Since vertical scan velocity reaches a practical maximum, and since an 
increase in vertical scan velocity is required to maintain forward overlap 206 
as airplane 102 velocity V increases, velocity V is limited to a maximum. 

The area of the scene detected by each pixel (i.e., the area projected 
by each pixel) of the FPA at any given instant is defined as the sample size. 
As noted above, the EO camera is scanned such that the projection of each 
pixel is swept across the scene in the cross-track direction. This scanning 
causes the scene to be imaged in strips as discussed above. 

As the camera is scanned, a scene image is focused onto the FPA. The 
FPA transforms this scene image into an electrical charge representation of the 
optical information. FPAs are well known to those of ordinary skill in the art. 
The electrical charge information in the FPA is read out periodically during 
an FPA read in a conventional manner. 

In a preferred embodiment the scene image is optical information in the 
visible spectrum (i.e., light). Other embodiments may be contemplated 
wherein the scene energy detected is of an alternative wavelength such as 
infrared. It will be obvious to one of ordinary skill in the art how to select 
detector technology, optical components, and filters to optimize system 
performance for the desired operational wavelength. 

The area covered by the swept projected change in angle (related to the 
ground) between FPA reads is called the ground sample distance (GSD). Thus 
each strip comprises several GSDs. GSD is a function of pixel size, the focal 
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length of the system, the scan velocity of the camera, and the slant range p jUnl . 
As the slant range p jUnl decreases, the GSD also decreases. Therefore, as slant 
range Psiant decreases, resolution increases. As a result, the GRD for a given 
scan velocity and FPA read rate decreases as well. 
5 The slant range p jUal between the aircraft and the sample decreases as 

the scan is made from the far-field point of scan to the near-field point of 
scan. However, there is no requirement for improved resolution (beyond the 
specified GRD) in the near-field scan. To assure performance at a given IIRS 
across the entire cross-track field of coverage 202, the specified GRD for a 

10 desired IIRS must be met at the worst-case point, which is the far-field point 
of scan. The improved resolution at the near-field point of scan yields a better 
IIRS value, but this is of no practical benefit when the specified IIRS value has 
already been met. 

The present invention, however, takes advantage of this increased near- 

15 field resolution performance to overcome the problem in conventional systems 
of limiting the scan rate. Since the GSD decreases in the near field, the 
camera can be scanned faster (at a given FPA read rate) to cover the same 
amount of GSD in the near field as was covered in the far field at the slower 
scanning rate. 

20 Faster scanning in the near field increases the overall scanning rate of 

the camera. As a result, for a given IIRS requirement and a given FPA read 
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rate, the velocity V of the aircraft (and the amount of cross-track coverage) 
can exceed that to which it was limited with conventional systems. 

5. Embodiments of the Invention 

As discussed above, the present invention relies on what would 
5 otherwise be increased resolution in the near field to allow the scan velocity 
of the camera to be increased as the scan moves into the near field. 
According to the present invention, imagery data collected using an increasing 
scan velocity (non-linear scanning), is corrected on the ground to remove the 
geometric effects of such non-linear scanning, FIG. 3 is a flow chart 
10 illustrating a preferred process according to the present invention. FIG. 7 is 
a block diagram illustrating a preferred system according to the present 
invention. The present invention will now be described with respect to FIG. 
3 and FIG. 7. 

In a step 302, calculations are performed to determine system operating 
15 parameters such as camera scan rate and FPA read rates for a desired IIRS 
and given operational parameters such as aircraft velocity V. These 
calculations are performed with no restrictions on scanning rate. An 
associated FPA read rate, required to meet performance specifications 
(particularly resolution) at the determined scan rate, is calculated. For faster 
20 camera scan rates, the FPA read rate must also be faster. At camera scan 
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rates above a certain value, the FPA must be read at a rate faster than the 
detector technology permits- This scan rate is called the 'threshold* rate. 

If the required FPA read rate is within system capabilities, the camera 
scan velocity can be constant throughout the scan. If, on the other hand, the 

5 desired FPA read rate is faster than the system can handle, the scan velocity 
must be varied as a function of time. In other words, to obtain scan velocities 
above threshold, non linear scanning will be used, thereby increasing the scan 
velocity as the camera scans the near-field. 

In a step 303, if the scan rate is above the threshold rate, a non-linear 

10 scan velocity is determined. 

In a step 304, the camera is scanned across the scene to be imaged at 
the scan velocity as determined in step 302 (below threshold) or in step 303 
(above threshold). Camera scanning can be accomplished using a rotating 
prism assembly 702, a movable camera mount, a rotating camera barrel, or 

15 a number of other scanning techniques. These techniques for scanning a 
camera across a scene are well known to those of ordinary skill in the art. 

In a step 306, as the camera is scanned across the scene, an image of 
the scene is focused on an FPA 704 of the system. FPA 704 converts the 
image (visual, infrared, electromagnetic, or the like) into electrical charge 

20 information 722. Electrical charge information 722 is processed by a main 
electronics unit 706 to provide a physical electronic signal 424A that 
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represents the image information focused onto the FPA. In other words, in 
step 306, the camera electronically 'photographs* the scene. 

Optimal data compression can be performed by data compression unit 
708 to compress the digital image data in electronic signal 424A. Data 
compression generates signal 424B. 

The image data are transmitted to the ground for processing as 
discussed above with reference to FIG. 4. If data compression was used the 
data must be decompressed after reception. 9 

As a result, the digital image data in electronic signal 424A is provided 
in electronic signal 426. 

In a step 308, digital image data in electronic signal 426 is processed 
in image processing unit 710 to provide an image of the scene as 
'photographed* by the camera. This processing involves converting the digital 
image information in electronic signal 426 into a visual image data signal 724 
that can be displayed on a monitor or other device, or provided on hard copy. 

In a step 310, the pixel aspect ratio of visual image data signal 724 is 
corrected by aspect ratio correction unit 712 to compensate for the effects of 
non-linear scanning. Pixel aspect ratio is the ratio of width to height. 

In a step 312, the image processed in step 308 is displayed on a 
monitor or other device 436, printed out in hard copy, or stored in a data base 
714 for later retrieval. 



The above discussion with reference to FIG. 3, provides a high-level 
discussion of the key steps of the present invention. Steps 302, 303 and 308 
are crucial to successful operation according to the present invention. These 
steps are described in greater detail in the subsections that follow. 

5 In steps 302 and 303 discussed above, the system and method 

according to the present invention determine the scan velocity desired. To 
meet performance objectives, the EO camera system according to the present 
invention collects its image data in such way that the angular aspect ratio of 
the pixels within a single image is not a constant 1: 1, but is variable (i.e., the 

10 scan velocity is non-linear). This results in an image which is distorted. 
However, it is desirable to provide a distortion-free final image with pixels 
whose angular aspect ratio is 1:1 everywhere within the image. According to 
the present invention, this is accomplished in step 308 by correcting the 
variable pixel aspect ratio in the EO-LOROPS Ground Exploitation System 

15 (GES). 

To perform this correction fast enough to keep up with the incoming 
data flow, the correction should remain computationally simple. However, the 
camera system angular scan velocity equations are typically relatively 
complex. To simplify the computations required, the camera scan equations 
20 are accurately approximated using a quadratic polynomial in depression angle. 
This allows the system to meet its performance objectives while at the same 
time yielding a computationally simple pixel aspect ratio correction procedure. 
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The following subsections of this application describe in greater detail 
the scan velocity determination of steps 302 and 303 and the pixel aspect ratio 
correction of step 308. 

Subsection 5.1 of this application presents the assumptions used in 
determining the scan velocity and the image correction relationships. 
Subsection 5.1 also discusses key system parameters. Subsection 5.2 of the 
application describes the equations for the exact angular scan velocity desired 
for each of the five modes, and the procedure used to approximate them with 
a quadratic polynomial in depression angle. Subsection 5.3 describes the 
procedure used to perform fast pixel aspect ratio correction. 

5.1 Assumptions and Key System Parameters 

Certain assumptions can be made to simplify the computations 
required. Described in this section is a list of the assumptions used in a 
preferred embodiment of the present invention. 

The first assumption used is that the earth is flat. This assumption 
greatly simplifies the scan velocity equations, and hence the correction 
equations. 

The second assumption is that there is no cross wind. This allows the 
assumption that the plane's motion is totally in the in-track direction and there 
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is no motion component in the cross-track direction. In other words, the 
second assumption is that there is no crab angle. 

The third assumption is that the aircraft is standing still for the duration 
of each scan. 

The fourth assumption is that for very small angles the value of the 
angle (radians) may be used instead of the sine of the angle. This fourth 
assumption is no more than a frequently used mathematical approximation. 

The final assumption is that the depression angle supplied when 
specifying a scan is always the maximum depression angle for the scan. 

The EO-LOROP camera system according to the present invention is 
capable of collecting imagery using five different modes over a wide 
performance envelope of aircraft velocity, altitude, and depression angle. 
Tables 2 and 3 list representative system parameters for an embodiment of the 
present invention. 
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Parameter 
Name 


Parameter 
Svmbol 

LJJ 111 


Minimum 
Value 


Maximum 
Value 


velocity 


V 


350 nmile/hour 


750 nmile/hour 


altitude 


H 


10,000 feet 


40,000 feet 


depression 
angle 




4° 


30 


ground range 


Aground 


5 nmile 


40 nmile 
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Parameter Name 


Parameter Symbol 


Value 


dead time 


t d 


0.4 second 


minimum cycle time 




0.78 second j 


maximum line rate 




2500 lines/second 


minimum line rate 


1 rutin 


400 lines/second 


focal length 


f 


66 inches 


detector width 


Wcet 


0.01 mm 


number of detectors 


"del 


12,002 


minified view size 




4096 pixels 


in-track coverage 


l C ov 


4.100277° 


detector angle 




5.96516 * 10* 
radians 



The equations below are equations for the camera parameters given 
above which are calculated as opposed to given: 

~ f 2 25 A mrnf in | 
'cov 85 2*arctan^ ~ J 
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del 



6^ = 



25Ammfin 



As noted, the present invention is described in terms of five modes. 
The five modes specified each have different parameters as listed in Table 4, 
below. Additional modes may be contemplated wherein alternative system 
parameters are specified. 



Table 4 



Mode 


Cross-Track 
Coverage x cov 


Line Rate 
Ir 


Overlap 


Ground 
Sample 
Distance GSD 


1 


8.0° 


variable 


0.12 


variable 


2 


variable 




0.12 


1.5 feet (18 in.) 


3 


variable 




0.12 


1.83 feet (22 
in.) 


4 


4.0° 


variable 


0.12 


variable 


5 


4.0° 


variable 


0.56 


variable 
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5.2 Determination of Scan Velocity and Line Rate 

As outlined above, the desired scan velocity and the non-linear scan 
velocity are determined in steps 302 and 303, respectively. Steps 302 and 303 
will be discussed in this section in more detail with reference to FIG. 5. 
5 In a step 502, the actual scan time used tj Used is determined. Scan time 

used t^urf is the amount of time in seconds during each scan cycle that the 
system actually scans the scene. Scan time used is a fraction of the scan time 
t, available for scanning the scene during the scan cycle. 

In a step 504, the system line rate lr is determined and the number of 
10 lines in the scan is calculated. System line rate lr is the rate at which lines of 
pixels are generated by the camera system. System line rate lr is a function 
of the mode selected. 

In a step 506, and end of frame time t^ is determined. End of frame 
time t^a is the time elapsed after the last complete image line has been 
15 generated during the scan. 

In a step 508, if the line rate lr is less than or equal to the maximum 
line rate Ir^, a constant scan velocity is used (step 509). If however, the line 
rate lr exceeds the maximum line rate Ir^, a non-linear scan velocity must be 
computed in a step 510 and the scan velocity can be increased during the scan, 
20 keeping X GSD constant. 



FIG. 6 is a flow chart illustrating the steps followed in determining the 
scan time used i sased in step 502. Referring to FIG. 6, in a step 602, slant 
rage is determined as: 



H 



sinCe^) 



In a step 604, the cycle time t c is computed. To compute cycle time, 
the in-track distance at the near field is calculated. At depression angle 0 dmax 
(near-field) the camera images an in-track distance equal to: 



The cycle time is simply chosen as the time it takes the aircraft to travel a 
fraction of this in-track distance. A fraction of this distance is used (as 
opposed to the entire distance) to provide forward overlap. If the cycle time 
t c is quicker than the time it takes to cover the in-track distance, the scanned 
strips will overlap. The fraction used is defined by (1-ov) where ov is a 




v* 



6Q76ftJnmile 
3600sec//rr 
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number less than 1 indicating the amount by which the images must overlap. 
For example ov = 0.12 provides a 12% overlap. Thus, the cycle time can be 
described as: 



t e = (1 -ov)*- 



V * 3600sec//ir 



In practical applications, there is a lower bound t^n on the cycle time 
t c due to the fact that the camera scanning mechanism has a maximum 
rotational velocity. Therefore, the computed value of t c must be compared to 
the lower bound t cinin . If t c < t cmin the desired overlap is sacrificed to 
achieve a reasonable cycle time using the following procedure: 

if ov > 0.12, 

set ov = 0.12 

and repeat the calculation. 

If t c is still < t cmin , 

or if ov < + 0.12 initially, 

t c — t cm ; n . 

The actual overlap achieved can then be calculated from the cycle time by 
rearranging the above equation to solve for ov. 
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In a step 606, the scan time is calculated. Given the cycle time % the 
scan time available t, is then calculated as: 

t, = t, - t d 

In a step 608, the actual scan time used t^^ is calculated as: 

truncatejtoJnteger(t s * 1000 msecfsoc) 
w 1000 msec/sec 

The scan time used is truncated to an integral number of milliseconds, in a 
preferred embodiment, the airborne system will update the camera scan 
velocity at 1 millisecond intervals. Alternative embodiments may be 
considered wherein the airborne system updates the camera scan velocity at 
other periodic or nonperiodic intervals. 

Once the scan time t,^ is determined in step 502, the line rate and the 
number of lines in the scan can be calculated in step 504. For modes 2 and 
3 the line rate is always lr,^, and the number of image lines in the scan will 
be 

"lines = truncate joJntegerCt^/lr). 

For modes 1, 4, and 5 it is desired to produce lines whose cross-track 
angle is that will fit into the cross-track coverage x^, rounded to an 
integer: 



n iwes = truncate jo Jnteger\ 



The line rate lr is next calculated so that n liae , lines will be produced, such that: 




If the line rate calculated exceeds Ir^ the line rate must be limited to 
Ir^, and the scan velocity varied non-linearly, such that: 

lr = min (irjr^). 

The number of lines generated during the scan is therefore: 

niines — truncate jojnteger (lr * t,^. 

The answer is truncated to an integer, discarding any partial line. 

The end-of-frame time determined in step 506 is therefore: 



40 



The non-linear scan velocity determined in step 510 can be determined 
using an exact solution or a polynomial approximation. These determinations 
are discussed in the subsections that follow. Subsection 5.2.1 describes exact 
solution scan equations for Modes 1,4 and 5. Subsection 5.2.2 describes 
polynomial approximation scan equations for Modes 1, 4, and 5. Subsections 
5.2.3 and 5.2.4 describe exact solution and polynomial approximation scan 
equations, respectively, for modes 2 and 3. 

5.2.1 Exact Solution Scan Equations for Modes 1, 4, and 5. 

This section presents the exact solution for the modes 1, 4, and 5 
angular scan velocity. 

If the desired line rate as calculated in Section 4 is less than or equal 
to lr^, the angular scan velocity of the camera is a constant throughout the 
scan and is given by: 

*>(0 - I* * lr. 

Scanning at this constant angular scan velocity will generate pixels with the 
desired constant angular aspect ratio. 6^ can be precisely calculated as 

@dmtn ~ 4bn*x " ^dll * ^laws* 
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As illustrated by step 508, if the desired line rate exceeds Ir^, the 
angular scan velocity of the camera will vary with time, and the pixels 
produced will have variable angular aspect ratios which will have to be 
corrected by the GES. 
5 To see how this will be done, it is helpful to first examine in detail the 

three key equations that define in-track GSD, i GSD , cross-track GSD, x GSD , and 
GSD: 

e dct 

ZVcn = k * 



lr 



""GSD 



GSD" — 1gSD * *GSD' 

For good performance over the entire scan (in terms of GSD), some 
10 resolution in the near field is sacrificed to obtain better resolution in the far 
field. To accomplish this, the scan velocity is varied in such a way as to keep 
Xc SD constant: 
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ae 



sm(QJ> 



GSD 



-h 



2*lr*t. 



Solving this equation for 30d/3t yields an angular scan velocity of: 



-In 



t*(f) = 



~dt 



2 * r. 



which simplifies to: 
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<a(r) = — *=A * sin(Q d ) 

Ot 



solving for A yields: 



2 * ^ 



5 



Solving the above equation for 6 d as a function of time and then 
differentiating to get w(t) yields: 



V 
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0.5 * (!n(l + cos{QJ»-ln(l-cos(e d ))) 



- C -t 



C = t - 



0.5 * + cos(6 d ))-ln(.l-cos(Q d ))) 



6/0 = arccos 



/x 36* 2*i4*e-^ (C - <) 

om = = 

w 3f (g-^c^a ♦ l 

In the above equations the sign of time has been reversed so that it is 
always non-negative and the scan goes from to 9^ as t increases from 
0. The constant of integration C can be easily computed by substituting t = 
0 and 6 d = 0^ into the equation for C. 6^ can then be precisely calculated 
5 by substituting t tnd into the equation for 0,,(t). 
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5.2.2 Polynomial Approximation Scan Equations for Modes 1, 4, 
and 5. 

To find a quadratic polynomial approximation to the exact solution for 
the angular scan velocity of modes 1, 4, and 5, three equations with three 
unknowns are set up. The equations are solved for the coefficients. Note, it 
is desirable to obtain an approximation in which t begins at 0 and increases 
regardless of whether the scan is a forward (0 dmax to 0 dmin ) or reverse (0 dmin 
to fldmax) scan. Therefore, in setting up the equations for a forward scan, time 
tp 0ly is used in the approximate equations and t exact = t^jy in the exact 
equations. In setting up the equations for a reverse scan, tp 0ly is used in the 
approximate equations and t exact = (t^a - tp 0ly ) is used for the exact 
equations. 

Three equations in three unknowns of the following form are desired: 

"exactCexact) = a * tpoly 2 + b * V*y + ° 

Set tQ = 0 and At = t^^ Pick values of tp 0ly « 0, Xq + At/2, and % + At; 
compute t exact as described above; set up the equations; and solve for a, b, and 
c. The result will be an approximate solution for o)(t) of the form 

«WW = ^W 2 + b * *Pob + ° 
in which t begins at \§ + At regardless of the scan direction. 

This method can be generalized to include constant angular pixel aspect 
ratio images by setting: a = 0, b = 0, and c = 0 dri * lr. 
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5.2.3 Exact Solution Scan Equations for Modes 2 and 3. 

This section presents the exact solutions for the modes 2 and 3 angular 
scan velocity. 

Camera modes 2 and 3 collect imagery in such a way that the GSD is 
5 constant throughout the image. The line rate lr is always lw, the angular 
scan velocity of the camera will vary with time, and the pixels produced will 
have variable angular aspect ratios which will have to be corrected by the 
GES. To illustrate how this is accomplished, first examine in detail the three 
key equations that define in-track GSD i^D. cross-track GSD x GSD , and GSD: 



sinQJ) 




dt lr 



GSD 



I. 



GSD 




10 To keep GSD constant throughout the scan x GS d can be varied to 

exactly compensate for changes in i CSD . However, at large depression angles 



this would lead to very small values of i GSD and correspondingly large values 
of x GSD . However, when x GSD gets very large a very small i GSD cannot be used 
to compensate. The system is therefore limited to the larger of i GSD and 
Thus, for large depression angles x GSD is held constant to maintain the 
desired GSD (see subsection 5.3 for the mode 2 and 3 values of GSD): 



N 2 



= GSD 



X GSD " ft * OSD = 




solving this equation for d6d/dt where: 



dfy = yfl * GSD * Ir 
8t h 



* sin(Q d ) 



yields: 



(*{i)=—±=A*sin(BJ thus: A = 



y/2 * GSD *lr 
h 
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Note that this is the same basic form as the exact solution for modes 
1, 4, and 5 with a different value for the constant A. Using the results from 
subsection 5.4 yields: 



0.5 *(ln(l +cos(B d )) -c<w(e rf ))) 



■C-t 



f = C + 



0.5 +cos(6 d )) -/«( -cos(d d ))) 



0.5 * +a«(e <f )) -ln( -«w(6 a ))) 



0 a (r) =arccos 



-A*ic-ty 



(D(r) = — ± =2 *.4 — 
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In the above equations the sign of time t has again been reversed so that t is 
always non-negative and the scan goes from 0^ to 0^ as t increases from 
0. The constant of integration C can be easily computed by substituting t = 
0 and 0 d = 6^ into the equation for C. 
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The above equations are used in modes 2 and 3 until the value of i GSD 
reaches x GSD /2. The depression angle at which this occurs is: 



_ X GSD 

l cso - — 



ft*0dct = fi + GSD 



e 



arcsin 



GSD 



It is important to note that the conditions which may arise are: 

^droax* ^dmw ^ ^dcutoff ^ ^dmim or ^dmin ^dcutoff* ^ ^dcutoff ^dmix then the 

5 equations above will not be used as the scan will be completely described by 
the equations presented below. If 0^ > we can solve for by 
substituting Cutoff into the equation for t above. If -> then the scan 
will be completely defined by the equations above, and 6^ can be found by 
substituting into the equation for 0 d (t). If t^ < t^ then 0^ > 0 dcwoft 

10 > 0^, the first part of the scan will be described by the equations above, 
and the second part by the equations presented below. 
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For e M > = Gdoun, x GSD is allowed to vary in order to maintain a 
constant GSD as follows: 

ioso * x GS0 = GSDJ 

36, i 
A*— * — 

h *** * § £ = GSD 2 

sinidj) sin(6 d ) 

03(0 = ir "ii * nrJ sm i d) 



o(t)=— 7=/4*«'n2(e < ,) 
at 




Solving the above equation for 6 d as a function of time, and then 
5 differentiating to get w(t), we get: 

L_ = c -< 



In the above equations the sign of time t has again been reversed so 
that t is always non-negative and the scan goes from 0^ to 0^ as t increases 
from 0. If 0 dculoff >. 0^ the constant of integration C can be easily computed 
by substituting t = 0 and 0 d = 0^ into the equation for C. If 0^ > O^tf 
> 0^ then the constant of integration C is computed by substituting t - 
and 0 d = 0^^ into the equation for C. 0^ can be found by substituting t^ 
into the equation for 0 d (t). 

5.2.4 Polynomial Approximation for Modes 2 and 3. 

To find a quadratic polynomial approximation to the exact solution for 
the modes 2 and 3 angular scan velocity three equations in three unknowns are 
set up and solved for the coefficients. If the scan is completely described by 
either the first set or the second set of mode 2 and 3 equations presented in 
Section 5.1 then only one set of equations must be solved. If however 0^ 
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> fldcuioff > 0dmin then the scan will consist of two parts: the first described by 
the constant cross-track GSD equations, and the second described by the 
variable cross-track GSD equations. Thus, two sets of equations will have to 
be solved to find polynomial approximations for each part of the scan. 
5 It is desirable to obtain an approximation in which t begins at 0 and 

increases regardless of whether the scan is a forward (0^ to 6^ or reverse 
(0dmin to 6^ scan. Therefore, in setting up the equations for a forward scan, 
use time t^y in the approximate equations and t cxact = t^ in the exact 
equations. In setting up the equations for a reverse scan, use t^ in the 
10 approximate equation and t^ = (t^ - t^) for the exact equations and, if 
the scan is a two-part scan as discussed above, set t cutoff = (ts Used - t^n). 

It is desired to have (possibly two sets of) three equations in three 
unknowns of the following form: 

"exact&xact) = * * W + *> * tpoly + ^ 

15 For one part scan, set to = 0 and At = t^; for a two-part scan, first solve 
the equations using to = 0 and At = t^^, then using to = t cutoff and At = (t^ 
- tcutoff)- P^k values of t^ = to, to + At/2, and to + At; compute t exact as 
described above; set up the equations; and solve for a, b, and c. 

For each set of equations the result will be an approximate solution for 

20 w(t) of the form: 
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in which t begins as to and increases to to + At regardless of the scan 
direction. For a two part scan, use the first polynomial if t < t^, the 
second if t > = t^. 

5.3 Pixel Aspect Ratio Correction Procedure 

As described above, in step 308 the electronic signal representing the 
imagery data are processed and the pixel aspect ratio corrected. The pixel 
aspect ratio correction will now be discussed in detail. This subsection 
presents the correction in 2 steps. The first step is a derivation of the 
correction and the second step is an implementation of the pixel aspect ratio 
correction procedure. 

5.3.1 Derivation 

The EO-LOROPS GES must produce minified view images via pixel 
averaging tt on-the-fly" (in real time) as data are received or played back from 
a digital tape recorder. The minified view must fit into a n^ x n^ pixel 
buffer and the pixels in the minified view must have an angular aspect ratio 
of 1:1. For an image collected using a constant angular scan velocity the 
computation of the reduction factor is relatively simple: 
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( mzxin^n^-l) + n minif - 1 ) 
>jtojinteger\ 



72 = truncate. 

avg 



However, for an image with variable angular pixel aspect ratios, the 
averaging factor in the line (cross-track) direction will vary because the angle 
represented by each line varies. What is desired is a function indicating for 
each line in the minified view image how many lines are to be averaged from 
the full resolution image. For two-part scans two such functions are required. 
To find this function, first compute n avg using: 



in place of n te : 

w( e<W e~ e<to,i " ,^-2)^-1 > 



n^ = truncate _to_integer 



n mimf 



This value of n avg is used as the reduction factor in the pixel (in-track) 
dimension. The angle corresponding to this number of pixels is: 
10 Atinuf = n avg * 0 del . 
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The next step is to find a third-order polynomial approximation of time 
t as a function of angle 0^ scanned since the start of the scan. Integrating the 
following: 

yields: 

WW'f * W» + f *W +C * 

For a one-part scan or the first part of a two-part forward scan, d = 0. 
For the second part of a two-part forward scan: 

For a reverse scan substitute (0 cutofr - 0^) for (0^ - 0 dcutoff ) and set t cutoff = 
(tw*d - tc^n). The coefficients a, b, and c are for the second part of the scan. 
Now, four equations in four unknowns are set up to find: 

<W - at * 0PO* 3 (W + b t * 0^ 2 (t^) + c, * 0^^) + d t 
Through a series of substitutions one can now find a function n te of line 
number in the minified view image 1^ that gives the number of lines in the 
full resolution image to be used to create one line in the minified view image: 
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(3 * a, * A6 mMf + 2 * b t ) * A0 minif * 9^ + 
((^ * AO^t + b t ) * + c.) * A0 niI , if 



where: 



and: 



0poly _ 'minif * A^minif 



these volumes can be substituted to arrive at: 

Win** + 1) " Win**) = 3 * a, * A^minif 3 * lr * l^ 2 + 

10 (3 * a, * Ad mia « + 2 * b t ) * A0 minif 2 * lr * l minif + 

((a, * + bd * Afl,,,^ + c.) * A6 miaiS * lr 

To express this equation in simpler terms, define: 

a te = 3 * a, * AW * lr 

b te = (3 * a, * A0 minif + 2 * b t ) * AU * lr 



c te - ((a, * A0 minif + b t ) * A0 mInif + q ) * A0 minif * Ir 
substituting these terms yields: 

5.3.2 Implementation 

When it is time to compute line number 1^ (zero relative) in the 
minified view image, the function n^l^f) is evaluated. Front-end electronics 
in the GES obtains n^ln^) full resolution lines, and averages down by n^ in 
the pixel (in-track) dimension and by n^O^jf) in the line (cross-track) 
dimension. Evaluation of n^I^) can be done without any multiplications 
within a loop as demonstrated by the following pseudo-code: 

nig « (alg * l_initial + big) * 1 initial ♦ clg 
inc - alg * (2 * l_initial ♦ 1) -►"big 

for ( 1 - l_initial; 1 < l_last; 1++) /* note letter 1, not number 1 */ 

/* at thie point nig is correct for the current value of 1 */ 
nig inc 
^ inc 2 * alg 

In general the value of n^l^f) will not be an integer, yet for a fast 
hardware implementation it is desired to use an integral number of full 
resolution image lines to construct each minified view line. Therefore as each 
minified view line is computed we must round n^l^) to the nearest integer 
and add the (positive or negative) error amount to the next value of n^l^f). 
In this way the corrected image will "track" an exactly interpolated correction 
to the nearest full resolution line number. The following pseudo-code 



accomplishes the angular pixel aspect ratio correction using shifted 32-bit 



integer arithmetic. 



* Set up 32 -bit integer numbers needed for aspect ratio correction. The 

* only floating point numbers are "theta's and flttemp. Thie would be 

* done by the airborne system and the various 32 -bit numbers put into the 

* pre-frame file (prf ) . The prf would contain: 
• 

* nlines 2 bytes 

* navg 1 byte 
nminif _part [2] 4 bytes 
nlg_part((23 8 bytes 
inc27_part [23 8 bytes 
inc38_part [2] 8 bytes 

inc3C 2a_partf2j 39 bytes 

* define SH 27 
Sdefine SH2 11 
^define MASK <-(2*SH-l)) 
Sdefine MASK2 (- (2*SH2-1) ) 
npart - 1 

if (one-part scan) . .,. 

nminif_part [03 « ceil ( (the ta_d_max - theta_d_mm) / del ta_theta_ma.nl f) 
else 

{ 

npart « 2 

if (forward scan) . . . n 

nminif_part [0] * ceil ( (the ta_d_max - theta_d_min) / delta_theta_mxmf ) 

nminif _part (03 » ceil ( (theta_cutof f - fcheta_d_mxn) / 
delta_theta_minif) 

nminif_j>art [[13 - ceil ( (theta_d_max - theta_d_min) / delta_theta_minif ) 
for ( p - 0; p < npart; p++ ) /• for each part of the scan */ 

^ lminif « p * nminif _part (p3 /* minified view line num at start of part */ 
nig part (p3 « round( ( (alg (p 3 * lminif ♦ blg[p3) * lminif * clg [p3 ) * 2 SH) 
flttemp - (alg(p3 * (2 * lminif + 1) + blg[p3 * 2~SH 
inc27_part [p3 - truncate (flttemp) 

inc38 part(p] - round ( (flttemp - inc27_part [pj ) * 2 SH2) 
inc38_2a_part [p3 - round (2 * alg(p3 * 2~<SH+SH2)) 

/* 

* Perform angular pixel aspect ratio correction- Thus is done by the GES 

* front-end electronics. 

for ( p - 0, lminif « 0, rem - 0; lminif < nminif _part (p3 ; P++ ) 

/* get parameters for this part of scan •/ 
nminif - nminif jpart [p] 
nig - ngljpartlpl 
inc27 * inc27 _part tp3 
inc38 - inc38_partlp) 
inc38_2a « inc38_2a_part Ip] 
/* create minified view lines •/ 
for ( ; lminif < nminif; lminif ) 

nlg_pluB_rem « nig + rem 

nlg_j>lu6_ren_plus_half * nlg_plus_rem ♦ 2 (SH-1) 
nlg_integer - nlg__plue_rem _j>lus_half » SH 
if ( nlg_integer < 1 7 

nlg_integer ■ 1 
/* at this point use navg, nlg_integer to do averaging */ 
rem - nlg_plus_rem - (nlg_plus_rem_plus_half & MASK) 
inc27 +« inc38 » SH2 
nig +* inc27 
inc38 inc38 & MASK2 

inc38 +« inc38 2a 

} 
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Appendix 



IMAGERY INTERPRETABILITY RATING SCALE (IIRS) 

RATING CATEGORY 0 

Useless for interpretation due to cloud cover, poor resolution, etc. 

RATING CATEGORY 1 

Ground Resolved Distance: Greater than 9 meters (>29.5 ft.) Q354 
inches) 

Detect the presence of larger aircraft at an airfield. 
Detect surface ships- 
Detect ports and harbors (including piers and warehouses) 
Detect railroad yards and shops. 
Detect coasts and landing beaches. 
Detect surface submarines. 

Detect armored artillery ground force training areas. 
Recognize urban areas. 
Recognize terrain. 
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RATING CATEGORY 2 

Ground Resolved Distance: 4.5 to 9 meters (14.45 to 29.5 ft.) 

(177 to 354 inches) 

Detect bridges. 

5 - Detect ground forces installations (including training areas, 

administration/barracks buildings, vehicle storage buildings, and 
vehicle parking areas. 

Detect airfield facilities (count accurately all larger aircraft by 
type, straight-wing and swept/delta-wing). 

10 - Recognize ports and harbors (including large ships and 

drydocks). 

RATING CATEGORY 3 

Ground Resolved Distance: 2.5 to 4.45 meter s (8.2 to 14.75 ft.) 

(98 to 177 inches) 

15 - Detect communications equipment (radio/radar). 

Detect supply dumps (POL/Ordnance). 

Detect and count accurately all straight-wing aircraft, all swept- 
wing aircraft and all delta-wing aircraft. 

Detect command and control headquarters. 

20 - Detect surface-to-surface and surface-to-air missile sites 

(including vehicles and other pieces of equipment). 

Detect land minefields. 

Recognize bridges. 

Recognize surface ships (distinguish between a cruiser and a 
25 destroyer by relative size and hull shape). 

Recognize coast and landing beaches. 

Recognize railroad yards and shops. 
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Identify airfield facilities. 

Identify urban areas. 

Identify terrain. 

RATING CATEGORY 4 

5 Ground Resolved Distance: 1.2 to 2.5 meters (3.94 to 8.2 ft.) 

(47.25 to 98 inches) 

Detect rockets and artillery. 
Recognize troop units. 

Recognize aircraft such as FAGOT/MIDGET when singly 
10 deployed). 

Recognize missile sites (SSM/SAM). Distinguish between 
missile types by the presence and relative position of wings and 
control fins. 

Recognize nuclear weapons components. 
15 - Recognize land minefields. 

Identify ports and harbors. 
Identify railroad yards and shops. 

Identify trucks at ground force installations as cargo, flatbed, 
or van. 

20 - Identify a KRESTA by the helicopter platform flush with the 

fantail, a KRESTA II by the raised helicopter platform (one 
deck level above fantail and flush with the main deck). 
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RATING CATEGORY 5 

Ground Resolved Distance: 0.75 to 1.2 meters (2.46 to 3.94 ft.) 

(29.53 to 47.25 inches 

Detect the presence of call letters or numbers and alphabetical 
country designator on the wings of large commercial or cargo 
aircraft (where alpha-numerics are 3 ft. high or greater). 

Recognize command and control headquarters. 

Identify a singly deployed tank at a ground forces installation 
as light or medium/heavy. 

Perform Technical Analysis on airfield facilities. 

Perform Technical Analysis on urban areas. 

Perform Technical Analysis on terrain. 

RATING CATEGORY 6 

Ground Resolved Distance: 40 to 75 centimeters (1.31 to 2.46 ft.) 

(15.75 to 7.87 inches) 

Recognize radio/radar equipment. 
Recognize supply dumps (POL/Ordnance). 
Recognize rockets and artillery. 
Identify bridges. 
Identify troop units. 

Identify FAGOT or MIDGET by canopy configuration when 
singly deployed. 

Identify the following ground force equipment; T-54/55 tank, 
BTR-50 armored personnel carrier, 57 mm AA gun. 

Identify by type, RBU installations (e.g. , 2500 series), torpedo 
tubes (e.g., 21 inch/53.34 cm), and surface-to-air missile 
launchers on a KANIN DDG, KRIVAC DDGSP, or KRESTA 
II. 
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Identify a ROMEO-cIass submarine by the presence of the 
cowling for the snorkel induction and the snorkel exhaust. 

Identify a WISKEY-cIass submarine by the absence of the 
cowling and exhaust, 

RATING CATEGORY 7 

Ground Resolved Distance: 20 to 40 centimeters (0.66 to 1.31 ft.) 

(7.87 to 15.75 inches) 

Identify radar equipment. 

Identify major electronics by type on a KILDEN DDGS or 
KASHIN DLG. 

Identify command and control headquarters. 
Identify land minefields. 

Identify the general configuration of an SSBN/SSGN submarine 
sail, to include relative placement of bridge periscope(s) and 
main electronics/navigation equipment. 

Perform Technical Analysis on ports and harbors. 

Perform Technical Analysis on railroad yards and shops. 

Perform Technical Analysis on roads. 

RATING CATEGORY 8 

Ground Resolved Distance: 10 to 20 centimeters (0.33 to 0.66 ft.) 

(3.94 to 7.87 inches) 

Identify supply dumps (POL/Ordnance). 

Identify rockets and artillery. 

Identify aircraft. 

Identify missile sites (SSM/SAM). 
Identify surface ships. 
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Identify vehicles. 

Identify surfaced submarines (including components such as 
ECHO II SSGN sail missile launcher elevator guide and major 
electronics/navigation equipment by type). 

Identify, on a KRESTA II, the configuration of major 
components of larger electronics equipment and smaller 
electronics by type. 

Identify limbs (arms, legs) on an individual. 

Perform Technical Analysis on bridges. 

Perform Technical Analysis on troop units. 

Perform Technical Analysis on coast and landing beaches. 

RATING CATEGORY 9 

Ground Resolved Distance: less than 10 centimeters (<0.33 ft.) 

(<3.94 inches) 

Identify in detail the configuration of a D-30 howitzer muzzle 
brake. 

Identify in detail on a KILDEN DDGS the configuration of 
torpedo tubes and AA gun mountings (including gun details). 

Identify in detail the configuration of an ECHO II SSGN sail 
including detailed configuration of electronics communications 
equipment and navigation equipment. 

Perform Technical Analysis on radio/radar equipment. 

Perform Technical Analysis on supply dumps (POL/Ordinance) . 

Perform Technical Analysis on rockets and artillery. 

Perform Technical Analysis on missile sites. 

Perform Technical Analysis on nuclear weapons components. 
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DEFINITIONS 



The following terms and definitions are used for the purpose of this 
appendix only. 



GROUND RESOLVED DISTANCE: 



10 



GROUND resolution: 



15 



20 



Ground Resolved Distance (GRD) is the 
minimum test target element resolved on the 
ground. With a system that produces a GRD 
of 1.0 foot, the smallest bar of the test target 
that can be distinguished in the best case has 
a physical width of 0.5 foot. (A Tri-bar test 
target was used to determine GRD and 
subsequently, to calibrate the Imagery 
Interpretability Rating Scale.) 

Ground Resolution, a term used in photo- 
interpretation, is a subjective numerical 
estimate of the limiting size of ground 
objects imaged on film. It does not require 
a test target for its determination and may 
not equate to Ground Resolved Distance. 
The degree to which an individual can 
detect, recognize, and identify ground 
objects leads to his estimate of ground 
resolution. 
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detection: 



recognition: 



30 



identification: 



In imagery interpretation, the discovering of 
the existence of an object but without 
recognition of the object. 

The determination by any means of the 
friendly or enemy character of the 
individuality of another, or of objects such 
as aircraft, ships, tanks, or of a phenomena 
such as communications or electronics 
patterns. 

In imagery interpretation, the discrimination 
between objects within a particular type or 
class. 



35 technical analysis: the ability to describe precisely a feature, 

object, or component imaged on film. 
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CLAIMS 



1. A system for performing long range, sector scan panoramic electro- 
optical reconnaissance of a scene, the reconnaissance performed at increased 
scan velocities for a given ground resolved distance and forward overlap, 
comprising: 

(1) a focal plane array configured to detect an image of the scene 
and to convert said image into an electronic charge 
representation of said image; 

(2) a main electronics unit, coupled to said focal plane array, 
configured to convert said electronic charge representation into 
a digital image data signal, wherein said digital image data 
signal is a digital representation of said image; 

(3) first means, coupled to said focal plane array, for focusing a 
portion of the scene onto said focal plane array, wherein said 
portion of the scene is defined by a projection of said focal 
plane array; 

(4) second means, coupled to said first means, for scanning said 
projection of said focal plane array across the scene at a non- 
linear scan velocity; 
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(5) third means, coupled to said second means, for determining 
said non-linear scan velocity; and 

(6) fourth means, coupled to said main electronics unit, for 
processing said digital image data signal to provide a visual 

5 image signal representing a corrected image, said corrected 

image having a pixel aspect ratio corrected to remove the 
effects of said non-linear scan velocity. 

2. The system of claim 1, wherein said second means is configured to 
deterndne said non-linear scan velocity using an exact solution, and wherein 

10 said exact solution determines said non-linear scan velocity such that a cross- 
track ground sample distance is kept constant throughout the scan. 

3. The system of claim 1, wherein said second means is configured to 
use a polynomial approximation solution to determine said non-linear scan 
velocity using a polynomial approximation. 



15 



4. The system at any of claims 1 to 3, vterein said focal plane array is a charge 
coupled device matrix, said first means is an optical camera, and said fourth 
means is a rotating camera barrel. 



68 



5. The systsn of any at the preceding alains, funtter acnpdsing an a ir fram e craft for 
supporting said focal plane array, said main electronics unit and said first, 

second and third means. 

6. The system of claim 5, further comprising an air-to-ground data link 
coupled between said focal plane array and said fourth means, for transmitting 
said electronic signal from said airborne craft to a ground station. 

7. Tte system cf ary at the preosding claims, furtter carprisdjxf a data base, mpicri to said 
main electronics unit, for storing digital image data signal. 

8. The system at any at the psaading claine, further c mpu sing rveecis, mpi<y| tr> <*a ifl 
fourth means, for displaying said corrected image. 

9. The system of an/ at the rxeaading claims, furtter c a t|y ising fifta mare, ocxpLsd tr> said 
focal plane array, for compressing said digital image data signal to form a 
compressed digital image data signal, and sixth means, coupled to said fifth 

means, for decompressing said compressed digital image data to reproduce 
said digital image data signal. 
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10. A method for optimizing long range, sector scan panoramic electro- 
optical reconnaissance of a scene from an airborne craft comprising the steps 
of: 

(1) determining a camera scan rate and a focal plane array read 
5 rate for a desired level of operational performance; 

(2) determining a non-linear scan velocity if said scan rate is above 
a threshold level; 

(3) scanning the camera across the scene at said non-linear scan 
velocity determined in step (2); 

10 (4) focusing an image of the scene scanned in said step (3) onto a 

focal plane array and converting the image into an electronic 

signal that represents the scene; 
(5) processing said electronic signal to generate a digital image of 

the scene, wherein said processing comprises a step of 
15 correcting a pixel aspect ratio of said digital image to remove 

effects of said non-linear scan velocity, 

11. The method of claim 10, further comprising the step of displaying said 
corrected image. 

12. The netird cf claim 10 or claim 11, tfjecein said step (1) a jiji ires the sfa^s of: 
20 ( a ) determining an actual scan time used; 
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(b) determining a system line rate; 

(c) calculating the number of lines in a scan, wherein the 
calculation is made using said actual scan time 
determined in said step (a) and said system line rate 
determined in said step (b); and 

(d) determining an end-of-frame time as a ratio of said 
number of lines in a scan to said line rate. 



The netted at claim 10 car claim 11, vteredn said step (1) acnp rfaes the steps of: 

(a) determining an actual scan time used; 

(b) determining a desired number of lines in a scan; 

(c) calculating a system line rate as a ratio of said number 
of lines determined in said step (b) to said actual scan 
time used determined in said step (a); and 

(d) determining an end-of-frame time as a ratio of said 
number of lines in a scan to said line rate. 

Ihe method of claim 12 ar claim 13, \tecein saxi step far cfetermirurxj said seen. 



used comprises the steps of: 

(i) determining a slant range; 



computing an in-track distance in a near-field, wherein 
said in-track distance is a function of said slant range 
determined in said step (i) 

computing a cycle time as a fraction of said in-track 
distance computed in said step (ii), wherein said fraction 
is determined to provide a desired amount of forward 
overlap; 

calculating a scan time as a function of said cycle time; 
and 

calculating said scan time used as a function of said 
scan time. 
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5 15. hb nettled aaoan31ng to ay at cOairrB 10 to 14, \terein said step (2) cxirpziaes the 
steps of: 

(a) computing a cross-track ground sample distance; 

(b) computing an angular scan velocity needed to keep said 
cross-track ground sample distance constant; 

!0 ( c ) calculating a camera depression angle as a function of 

time based on said angular scan velocity; and 
(d) differentiating said depression angle calculated in said 
step (c) to obtain a non-linear scan velocity. 

16 . The i rellixl aooc a ri iiig to ay one at claims 10 to 14, therein said step (2) aenpdses the 
15 steps of: 

(a) calculating a cross-track ground sample distance 
required to maintain a constant overall ground sample 
distance; 

(b) holding said cross-track ground sample distance constant 
20 for large depression angles; 
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(c) computing an angular scan velocity required to maintain 
said cross-track ground sample distance; 

(d) calculating a camera depression angle as a function of 
time based on said angular scan velocity; and 

(e) differentiating said depression angle calculated in said 
step (c) to obtain a non-linear scan velocity. 



17. Ihe retire! awarding to ary of claims 10 to 16, vterein said step (2) crnprises the 
step of using a quadratic polynomial approximation to determine a non-linear 
scanning velocity. 

18. The netted amending to any of claims 10 to 17, Wierein said step (5) crrrpripes the 
steps of: 

(a) deriving a correction for said pixel aspect ration of said 
digital image; and 

(b) implementing said correction to correct said pixel aspect 
ratio. 

19. The method according to claim 18, wherein said step (a) comprises the 
steps of: 

(i) computing an averaging factor to produce minified view 
images; 
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(ii) computing a line group angle as the product of said 
averaging factor and a detector angle; 

(iii) determining a third-order polynomial approximation of 
time as a function of an angle scanned since the scan 
was started; and 

(iv) determining a full resolution line group based on said 
line group angle and using said third-order polynomial; 
and 

said step (b) comprises the steps of: 

(i) evaluating said full resolution line group; 

(ii) rounding said full resolution line group to a nearest 
integer value; and 

(iii) adding the amount of the rounding to a next value for 
said full resolution line group. 

J 0* A system for performing long range, sector scan panoramic electro- 
optical reconnaissance of a scene from an airborne craft, the reconnaissance 
performed at increased scan velocities for a given ground resolved distance, 
and forward overlap, comprising: 

first means for detecting an image of a portion of the scene and 
converting said image into a digital image data signal, wherein 
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said digital image data signal is a digital representation of said 
image; 

second means, coupled to said first means, for focusing said 
portion of the scene onto said first means, and for scanning said 
portion of the scene across the entire scene at a non-linear scan 
velocity; and 

third means, coupled to said first means, for processing said 
digital image data signal to provide a visual image data signal 
representing a corrected image, said corrected image having a 
pixel aspect ratio corrected to remove the effects of said non- 
linear scan velocity. 

21 . The system of claim 20 , further comprising fourth means, coupled to 
said second means, configured to use an exact solution to determine said non- 
linear scan velocity, and wherein said exact solution determines said non-linear 
scan velocity such that a cross-track ground sample distance is constant 
throughout the scan. 

22. The system of claim 20 further comprising fourth means, coupled to 
said second means, configured to use a polynomial approximation solution to 
determine said non-linear scan velocity. 
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23. The system of any of claims 20 to 22 , further 
comprising means, coupled to said first means, for 
transmitting said first electronic signal from the airborne 
craft to the ground. 

24. The system of any of claims 20 to 23, further 
comprising means, coupled to said first means, for storing 
said digital image data signal. 

25. The system of any of claims 20 to 24, further 
comprising means, coupled to said third means, for 
displaying said corrected image. 

26. The system of any of claims 20 to 25, further 
comprising means, coupled to said focal plane array, for 
compressing said digital image data signal to form a 
compressed digital image data signal, and fifth means, 
coupled to said fourth means, for decompressing said 
compressed digital image data signal. 

27. A system for performing long range, sector scan 
panoramic electro-optical reconnaissance of a scene, the 
reconnaissance performed at increased scan velocities for 
a given ground resolved distance and forward overlap 
substantially as hereinbefore described with reference to 
any of the examples shown in the accompanying drawings. 

28. A method for optimizing long range, sector scan 
panoramic electro-optical reconnaissance of a scene from an 
airborne craft substantially as hereinbefore described with 
reference to any of the examples shown in the accompanying 
drawings. 
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